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DETAILED ACTION 

1. Claims 2-6, 8-13 and 15-27 have been examined. Application 10/023,886 
(Information providing server, client, information providing system processing method, 
recording medium recording a program, and advertisement providing method) has a 
filing date 12/21/2001 and foreign priority 12/25/2000. 

Response to Amendment 

2. In response to Non Final Rejection filed 05/06/2008, the Applicant filed an 
Amendment on 05/06/2008, which amended claims 2, 4, 5, 8, 9, 11, 12, 15-17, 19, 21- 
25-27 and cancel claims 7 and 14. 

Claim Objections 

3. Claim 25 is objected to because is marked as (currently amended), however, 
there are no marking in the claims. 

Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed 
or described as set forth in section 102 of this title, if the differences between the 
subject matter sought to be patented and the prior art are such that the subject 
matter as a whole would have been obvious at the time the invention was made 
to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was 
made. 

Claims 2, 4, 5, 8-13 and 15-27 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Hirono (US 6,882,348) in view of Herf (US 6,734,873) and further in 
view of Neven (US 6,948,1 31 ). 
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As per claim 2, Hirono teaches: 

An information providing server comprising: 

image data storage means for storing image data generated based on image 
information in which a physical position is clearly described and in which a same area is 
photographed from different locations, and storing positional information showing the 
position of said image data (see figure 5); 

advertisement placing information storage means for storing advertisement 
placing information including at least a placing period and a placed location of an 
advertisement by adding an ID for each advertisement (see figure 5); 

image synthesizing means for reading the image data from said image data 
storage means based on a viewpoint of a user, reading from said advertisement placing 
information storage means said advertisement placing information having said placing 
period including the current date data and said placed location included in said image 
data read based on said viewpoint of a user, and synthesizing said read image data 
with said advertisement placing information to generate synthesis image data (see 
figure 5); 

advertisement contract storage means for storing contract information including 
an ID added to said advertisement placing information, the name of an advertisement 
placing person who desires to place an advertisement, and a contract money amount 
(see col 7, lines 45-50); and 

advertisement contract means for receiving an advertisement placing request 
from said advertisement placing person (see col 7, lines 45-50) executing a contract 
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process of said advertisement placing request upon reception of said advertisement 
placing request from said advertisement placing person and updating said 
advertisement placing information stored in said advertisement placing information 
storage means based on said contract information stored in said advertisement contract 
storage means (see col 7, lines 45-50). 

user data storage means for recording the ID of a user and the viewpoint position 
of said user (see col 6, lines 1-10); 

other users maximum display threshold value storage means for storing an other 
users maximum display threshold value for defining a threshold value to display the 
maximum number of viewpoint positions of other users when displaying the viewpoint 
positions of the other users (see col 9, lines 35-45 "marker"); 

user position display means for adding a user position mark showing the user to 
the viewpoint position of the user in said image stereoscopic representation provided to 
said user, fetching from data stored in said user data storage means the viewpoint 
positions of said other users and said user IDs up to said threshold value defined by 
said other users maximum display threshold value in the order close to the viewpoint 
position of said user, and adding other users position marks showing said other users to 
said viewpoint positions (see col 9, lines 30-45; position marker); and 

Hirono fails to teach that said image data is a three-dimensional image data, the 
user data storage means also storing Ids of other users and viewpoint positions of the 
other users and interaction connection means for regarding, as persons to interact with, 
the user IDs of said other users corresponding to given specified other users position 
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marks when said user specifies said given other users position marks and starting an 
interacting function program to provide connection. However, Herf teaches a system of 
displaying three-dimensional images in a browser (see col 7, lines 30-50), DeLorme 
teaches a system that allow users sightsee in three dimensions (see col 15, lines 45-50) 
and also teaches a system that records all the locations of users in map (see figure 2; 
col 21, lines 55-67) and Neven teaches that it is old and well known in the 
communication art to allow users to communicate in real time with other users that are 
online (see Neven col 7, lines 50-63). Therefore, it would have been obvious to a 
person of ordinary skill in the art at the time the application was made, to know that 
Hirono would use the Herf and DeLorme 's system to presents three-dimensional image 
to a user's browser in order that said user has a better browsing experience and would 
allow users to communicate with other users that are online at said users' location, as it 
is old and well known to do so, as taught by Neven . 

wherein the synthesis image data includes the advertisement placed at a 
particular location corresponding to the placed location in a image corresponding to the 
image (see Hirono col 7, lines 20-50). Hirono fails to teach that said image data is a 
three-dimensional image data. However, Herf teaches a system of displaying three- 
dimensional images in a browser (see col 7, lines 30-50). Therefore, it would have been 
obvious to a person of ordinary skill in the art at the time the application was made, to 
know that Hirono would use the Herts system to presents three-dimensional image to a 
user's browser in order that said user has a better browsing experience. 

As per claim 4, Hirono teaches: 
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An information providing server comprising: 

image data storage means for storing three-dimensional image data generated 
based on image information in which a physical position is clearly described and in 
which the same area is photographed from different locations, and storing positional 
information showing the position of said image data (see figure 5); 

advertisement placing information storage means for storing advertisement 
placing information including at least a placing period and a placed location of an 
advertisement by adding an ID for each advertisement (see col 8, lines 1-15) the 
advertisement placing information being provided by an advertisement placing person 
(see col 7, lines 45-50); 

user data storage means for recording the ID of a user and the viewpoint position 
of said user (see col 6, lines 1-11); 

other users maximum display threshold value storage means for storing an other 
users maximum display threshold value for defining a threshold value to display the 
maximum number of viewpoint positions of other users when displaying the viewpoint 
positions of the other users (see col 9, lines 35-45 "marker"); 

user position display means for adding a user position mark showing the user to 
the viewpoint position of the user in said image stereoscopic representation provided to 
said user, fetching from data stored in said user data storage means the viewpoint 
positions of said other users and said user IDs up to said threshold value defined by 
said other users maximum display threshold value in the order close to the viewpoint 
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position of said user, and adding other users position marks showing said other users to 
said viewpoint positions (see col 9, lines 30-45; position marker); and 

Hirono fails to teach that said image data is a three-dimensional image data and 
the user data storage means also storing Ids of other users, viewpoint positions of the 
other users and interaction connection means for regarding, as persons to interact with, 
the user IDs of said other users corresponding to given specified other users position 
marks when said user specifies said given other users position marks and starting an 
interacting function program to provide connection. However, Herf teaches a system of 
displaying three-dimensional images in a browser (see col 7, lines 30-50), DeLorme 
teaches a system that allow users sightsee in three dimensions (see col 15, lines 45-50) 
and also teaches a system that records all the locations of users in map (see figure 2; 
col 21, lines 55-67) and Neven teaches that it is old and well known in the 
communication art to allow users to communicate in real time with other users that are 
online (see Neven col 7, lines 50-63). Therefore, it would have been obvious to a 
person of ordinary skill in the art at the time the application was made, to know that 
Hirono would use the Herf and DeLorme 's system to presents three-dimensional image 
to a user's browser in order that said user has a better browsing experience and would 
allow users to communicate with other users that are online at said users' location, as it 
is old and well known to do so, as taught by Neven . 

image synthesizing means for reading the image data from said three- 
dimensional image data storage means based on viewpoint of a user, reading from said 
advertisement placing information storage means said advertisement placing 
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information having said placing period including the current date data and said placed 
location included in said image data read based on said viewpoint of a user, and 
synthesizing said read image data with said advertisement placing information to 
generate synthesis image data (see figure 5); 

and image browser means for generating, based on a desired viewpoint 
position, a rendered image rendered from said synthesis image data outputted from 
said image synthesizing means (see figure 5); wherein the synthesis image data 
includes the advertisement placed at a particular location corresponding to the placed 
location in a image corresponding to the image (see Hirono col 7, lines 20-50). Hirono 
fails to teach that said image data is a three-dimensional image data. However, Herf 
teaches a system of displaying three-dimensional images in a browser (see col 7, lines 
30-50). Therefore, it would have been obvious to a person of ordinary skill in the art at 
the time the application was made, to know that Hirono would use the Herf 's system to 
presents three-dimensional image to a user's browser in order that said user has a 
better browsing experience. 

As per claim 5, Hirono teaches: 

An information providing server comprising: 

image data storage means for storing image data generated based on image 
information in which a physical position is clearly described and in which the same area 
is photographed from different locations, and storing positional information showing the 
position of said image data (see figure 5); 
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advertisement placing information storage means for storing advertisement 
placing information including at least a placing period and a placed location of an 
advertisement by adding an ID for each advertisement (see col 14, lines 45-60) the 
advertising placing information being provided by an advertisement placing person (see 
col 7, lines 45-50); 

image synthesizing means for reading the image data from said image data 
storage means based on viewpoint of a user, reading from said advertisement placing 
information storage means said advertisement placing information having said placing 
period including the current date data and said placed location included in said image 
data read based on said viewpoint of a user, and synthesizing said read image data 
with said advertisement placing information to generate synthesis image data (see 
figure 5); 

image browser means for generating, based on a desired viewpoint position, a 
rendered image rendered from said synthesis image data outputted from said image 
synthesizing means (see figure 5); 

advertisement contract storage means for storing contract information including 
an ID added to said advertisement placing information, the name of an advertisement 
placing person who desires to place an advertisement, and a contract money amount 
(see col 7, lines 45-50); and advertisement contract means for receiving an 
advertisement placing request from said advertisement placing person (see col 7, lines 
45-50), executing a contract process of said advertisement placing request upon 
reception of said advertisement placing request from said advertisement placing person 
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and updating said advertisement placing information stored in said advertisement 
placing information storage means based on said contract information stored in said 
advertisement contract storage means (see col 7, lines 45-55); wherein the synthesis 
image data includes the advertisement placed at a particular location corresponding to 
the placed location in a image corresponding to the image (see Hirono col 7, lines 20- 
50). Hirono fails to teach that said image data is a three-dimensional image data. 
However, Herf teaches a system of displaying three-dimensional images in a browser 
(see col 7, lines 30-50). Therefore, it would have been obvious to a person of ordinary 
skill in the art at the time the application was made, to know that Hirono would use the 
Herf s system to presents three-dimensional image to a user's browser in order that said 
user has a better browsing experience. 

user data storage means for recording the ID of a user and the viewpoint position 
of said user (see col 6, lines 1-10); 

other users maximum display threshold value storage means for storing an other 
users maximum display threshold value for defining a threshold value to display the 
maximum number of viewpoint positions of other users when displaying the viewpoint 
positions of the other users (see col 9, lines 35-45 "marker"); 

user position display means for adding a user position mark showing the user to 
the viewpoint position of the user in said image stereoscopic representation provided to 
said user, fetching from data stored in said user data storage means the viewpoint 
positions of said other users and said user IDs up to said threshold value defined by 
said other users maximum display threshold value in the order close to the viewpoint 
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position of said user, and adding other users position marks showing said other users to 
said viewpoint positions (see col 9, lines 30-45; position marker); and 

Hirono fails to teach that said image data is a three-dimensional image data and 
the user data storage means also storing Ids of other users and viewpoint positions of 
the other users and interaction connection means for regarding, as persons to interact 
with, the user IDs of said other users corresponding to given specified other users 
position marks when said user specifies said given other users position marks and 
starting an interacting function program to provide connection. However, Herf teaches a 
system of displaying three-dimensional images in a browser (see col 7, lines 30-50) and 
DeLorme teaches a system that allow users sightsee in three dimensions (see col 15, 
lines 45-50) and also teaches a system that records all the locations of users in map 
(see figure 2; col 21 , lines 55-67) and Neven teaches that it is old and well known in the 
communication art to allow users to communicate in real time with other users that are 
online (see Neven col 7, lines 50-63). Therefore, it would have been obvious to a 
person of ordinary skill in the art at the time the application was made, to know that 
Hirono would use the Herf and DeLorme 's system to presents three-dimensional image 
to a user's browser in order that said user has a better browsing experience and would 
allow users to communicate with other users that are online at said users' location, as it 
is old and well known to do so, as taught by Neven . 

As per claim 8, Hirono teaches: 

An information providing server comprising: 
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image data storage means for storing image data generated based on image 
information in which a physical position is clearly described and in which the same area 
is photographed from different locations, and storing positional information showing the 
position of said image data (see figure 5); 

advertisement placing information storage means for storing advertisement 
placing information including at least a placing period and a placed location of an 
advertisement by adding an ID for each advertisement (see col 14, lines 35-60); 

the advertisement placing information being provided by an advertisement 
placing person (See col 7, lines 45-50); 

image synthesizing means for reading the image data from said three- 
dimensional image data storage means based on browsing location specification 
information, reading from said advertisement placing information storage means said 
advertisement placing information having said placing period including the current date 
data and said placed location included in said image data read based on said view point 
of a user, and synthesizing said read image data with said advertisement placing 
information to generate synthesis image data (see figure 5); 

image browser means for generating, based on a desired viewpoint position, a 
rendered image rendered from said synthesis image data outputted from said image 
synthesizing means (see col 14, lines 45-60); 

landmark position storage means for storing landmark information including 
langrage representation information about the name or contents corresponding to a 
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landmark existing in said image data and positional information on the position of said 
landmark (see figure 12); and 

search engine means for searching for page data including the contents related 
to a keyword from a set of page data when said keyword is given as input and 
generating link information to said page data (see col 14, lines 45-60), 

wherein said image browser means refers to said positional information included 
in said landmark information of said landmark position storage means when related 
information presentation related to the position is commanded, specifies the 
corresponding landmark information, outputs said language representation information 
of the landmark information to said search engine means, and allows said search 
engine means to generate link information of page data related to the landmark 
information (see figure 12). wherein the synthesis image data includes the 
advertisement placed at a particular location corresponding to the placed location in a 
image corresponding to the image (see Hirono col 7, lines 20-50). Hirono fails to teach 
that said image data is a three-dimensional image data. However, Herf teaches a 
system of displaying three-dimensional images in a browser (see col 7, lines 30-50). 
Therefore, it would have been obvious to a person of ordinary skill in the art at the time 
the application was made, to know that Hirono would use the Herf s system to presents 
three-dimensional image to a user's browser in order that said user has a better 
browsing experience. 

user data storage means for recording the ID of a user and the viewpoint position 
of said user (see col 6, lines 1-10); 
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other users maximum display threshold value storage means for storing an other 
users maximum display threshold value for defining a threshold value to display the 
maximum number of viewpoint positions of other users when displaying the viewpoint 
positions of the other users (see col 9, lines 35-45 "marker"); 

user position display means for adding a user position mark showing the user to 
the viewpoint position of the user in said image stereoscopic representation provided to 
said user, fetching from data stored in said user data storage means the viewpoint 
positions of said other users and said user IDs up to said threshold value defined by 
said other users maximum display threshold value in the order close to the viewpoint 
position of said user, and adding other users position marks showing said other users to 
said viewpoint positions (see col 9, lines 30-45; position marker); and 

Hirono fails to teach that said image data is a three-dimensional image data and 
the user data storage means also storing Ids of other users and viewpoint positions of 
the other users and interaction connection means for regarding, as persons to interact 
with, the user IDs of said other users corresponding to given specified other users 
position marks when said user specifies said given other users position marks and 
starting an interacting function program to provide connection. However, Herf teaches a 
system of displaying three-dimensional images in a browser (see col 7, lines 30-50) and 
DeLorme teaches a system that allow users sightsee in three dimensions (see col 15, 
lines 45-50) and also teaches a system that records all the locations of users in map 
(see figure 2; col 21 , lines 55-67) and Neven teaches that it is old and well known in the 
communication art to allow users to communicate in real time with other users that are 
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online (see Neven col 7, lines 50-63). Therefore, it would have been obvious to a 
person of ordinary skill in the art at the time the application was made, to know that 
Hirono would use the Herf and DeLorme 's system to presents three-dimensional image 
to a user's browser in order that said user has a better browsing experience and would 
allow users to communicate with other users that are online at said users' location, as it 
is old and well known to do so, as taught by Neven . 

As per claim 9, Hirono teaches: 

An information providing server comprising: 

image data storage means for storing image data generated based on image 
information in which a physical position is clearly described and in which the same area 
is photographed from different locations, and storing positional information showing the 
position of said image data (see figure 12); 

advertisement placing information storage means for storing advertisement 
placing information including at least a placing period and a placed location of an 
advertisement by adding an ID for each advertisement (see col 6, lines 1-25); 

the advertisement placing information being provided by an advertisement 
placing person (see col 7, lines 45-50); 

image synthesizing means for reading the image data from said image data 
storage means based on viewpoint of a user, reading from said advertisement placing 
information storage means said advertisement placing information having said placing 
period including the current date data and said placed location included in said image 
data read based on said viewpoint of a user, and synthesizing said read image data 
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with said advertisement placing information to generate synthesis image data (see 
figure 12); 

image browser means for generating, based on a desired viewpoint position, a 
rendered image rendered from said synthesis image data outputted from said image 
synthesizing means (see figure 12); 

landmark position storage means for storing landmark information including a 
language representation about the name or contents corresponding to a landmark 
existing in said image data and positional information on the position of said landmark 
(see figure 12); 

search engine means for searching for page data including the contents related 
to a keyword from a set of page data when said keyword is given as input and 
generating link information to said page data (see col 14, lines 45-60); 

user stay landmark storage means for storing a landmark stay record including 
said landmark where a user stays and the stay time at the landmark (see figure 12); 

all movement history storing means for recording a movement history including 
the viewpoint position of said user and the time (see figure 12); 

stay time threshold value storage means for storing a stay time threshold value 
showing time to stay around a landmark necessary for judging that said user is 
interested in said landmark (see figure 12); 

range inside and outside judgment distance storage means for storing range 
inside and outside judgment distance information showing definition of the peripheral 
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position from the position of a landmark necessary for judging that said user is 
interested in said landmark (see figure 12); 

stay time calculation means for extracting said landmark information from said 
landmark position storage means, using positional information of the landmark 
information and said range inside and outside judgment distance information stored in 
said range inside and outside judgment distance storage means to extract from said all 
movement history storing means said movement history when the viewpoint position of 
said user is within a surrounding area of said landmark defined by said range inside and 
outside judgment distance information from positional information included in said 
extracted landmark position, and using the extracted movement history to calculate the 
first time and the last time in which the viewpoint position of said user is within said 
surrounding area of said landmark (see figure 12); 

and user stay landmark judgment means for comparing a stay time which is a 
difference between said last time and said first time calculated by said stay time 
calculation means with said stay time threshold value stored in said stay time threshold 
value storage means, and when said stay time is above said stay time threshold value, 
using said language representation information of said landmark information to store 
said landmark name and said stay time into said user stay landmark storage means, 
wherein said image browser means refers to said positional information included in said 
landmark information of said landmark position storage means when related information 
presentation related to the position is commanded, specifies the corresponding 
landmark information, outputs said language representation information of the landmark 
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information to said search engine means, and allows said search engine means to 
generate link information of page data related to the landmark information (see figure 
12). wherein the synthesis image data includes the advertisement placed at a particular 
location corresponding to the placed location in a image corresponding to the image 
(see Hirono col 7, lines 20-50). Hirono fails to teach that said image data is a three- 
dimensional image data. However, Herf teaches a system of displaying three- 
dimensional images in a browser (see col 7, lines 30-50). Therefore, it would have been 
obvious to a person of ordinary skill in the art at the time the application was made, to 
know that Hirono would use the Herf s system to presents three-dimensional image to a 
user's browser in order that said user has a better browsing experience. 

user data storage means for recording the ID of a user and the viewpoint position 
of said user (see col 6, lines 1-10); 

other users maximum display threshold value storage means for storing an other 
users maximum display threshold value for defining a threshold value to display the 
maximum number of viewpoint positions of other users when displaying the viewpoint 
positions of the other users (see col 9, lines 35-45 "marker"); 

user position display means for adding a user position mark showing the user to 
the viewpoint position of the user in said image stereoscopic representation provided to 
said user, fetching from data stored in said user data storage means the viewpoint 
positions of said other users and said user IDs up to said threshold value defined by 
said other users maximum display threshold value in the order close to the viewpoint 
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position of said user, and adding other users position marks showing said other users to 
said viewpoint positions (see col 9, lines 30-45; position marker); and 

Hirono fails to teach that said image data is a three-dimensional image data and 
the user data storage means also storing Ids of other users and viewpoint positions of 
the other users and interaction connection means for regarding, as persons to interact 
with, the user IDs of said other users corresponding to given specified other users 
position marks when said user specifies said given other users position marks and 
starting an interacting function program to provide connection. However, Herf teaches a 
system of displaying three-dimensional images in a browser (see col 7, lines 30-50) and 
DeLorme teaches a system that allow users sightsee in three dimensions (see col 15, 
lines 45-50) and also teaches a system that records all the locations of users in map 
(see figure 2; col 21, lines 55-67) interaction connection means for regarding, as 
persons to interact with, the user IDs of said other users corresponding to given 
specified other users position marks when said user specifies said given other users 
position marks and starting an interacting function program to provide connection 

As per claim 1 0, Hirono teaches: 

all information search command history storing means for recording an 
information presentation command history including the landmark name in which said 
user commands related information presentation related to the position and the time 
(see figure 12; col 6, lines 15-30); and 

user behavior record presentation means for outputting said landmark stay 
record of said user stay landmark storage means or said information presentation 
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command history of said all information search command history storing means when a 
manager commands to present the summing result (see col 6, lines 15-30). 
5. Claims 3 and 6 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Hirono (US 6,882,348) in view of Herf (US 6,734,873) and further in view of DeLorme 
(US 5,848,373). 

As per claim 3, Hirono teaches: 

image data conversion means for generating image data based on image 
information in which the same area is photographed from different locations and in 
which a physical position is clearly described (see fig 5). Hirono fails to teach that said 
image data is a three-dimensional image data and that wherein the synthesis three 
dimensional image data is provided as part of a three-dimensional aerial sightseeing 
service that is provided to the user for free. However, Herf teaches a system of 
displaying three-dimensional images in a browser (see col 7, lines 30-50) and DeLorme 
teaches presenting a three-dimensional sightseeing system (see col 15, lines 45-50). 
Therefore, it would have been obvious to a person of ordinary skill in the art at the time 
the application was made, to know that Hirono would use the Herf and the DeLorme 
system to presents three-dimensional image to a user's browser in order that said user 
has a better browsing experience. 

As per claim 6, Hirono teaches: 

image data conversion means for generating image data based on image 
information in which a physical position is clearly described in which the same area is 
photographed from different locations described (see figure 5). Hirono fails to teach that 
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said image data is a three-dimensional image data and wherein the synthesis three- 
dimensional image data is provided as part of a three-dimensional aerial sightseeing 
service that is provided to the user for free. However, Herf teaches a system of 
displaying three-dimensional images in a browser (see col 7, lines 30-50) and DeLorme 
teaches presenting a three-dimensional sightseeing system (see col 15, lines 45-50). 
Therefore, it would have been obvious to a person of ordinary skill in the art at the time 
the application was made, to know that Hirono would use the Herf and the DeLorme 
system to presents three-dimensional image to a user's browser in order that said user 
has a better browsing experience. 

Claims 11-13 and 15-27 contains the same limitation as claims 2-6 and 8-10 
therefore, the same rejection is applied. 

Response to Arguments 

6. Applicant's arguments with respect to claims 2-6, 8-13 and 15-27 have been 
considered but are moot in view of the new ground(s) of rejection. 

Conclusion 

7. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to DANIEL LASTRA whose telephone number is 571-272- 
6720 and fax 571-273-6720. The examiner can normally be reached on 9:30-6:00. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, ERIC W. STAMBER can be reached on 571-272-6724. The official Fax 
number is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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